Serial No. 09/495,175 



Attorney Ref: 2000-0025 



AMENDMENT 

This listing of claims will replace all prior versions, and listings, of claims in 
the application: 
Listing of Claims; 
1 - 24. (cancelled) 

25. (currently amended) A computer-implemented method for generating 
description records from multimedia content, comprising: 

identifying muhimedia types in muhimedia content; 

extracting multimedia objects to generate multimedia object descriptions from 
the muhimedia content for at least one of the muhimedia types; 

generating, from the multimedia object descriptions, non-hierarchical entity 
relation graph descriptions for at least one of the multimedia types , wherein the non- 
hierarchical entity relation graph descriptions are associated with communication 
between multimedia objects ; and 

integrating the muhimedia object descriptions and the entity relation graph 
descriptions to generate at least one description record to represent content embedded 
within the muhimedia content. 

26. (Previously Presented) The method of claim 25, further comprising generating, 
from the muhimedia object descriptions, muhimedia object hierarchy descriptions by 
object hierarchy construction and extraction processing, for at least one of the 
multimedia types. 

27. (Previously Presented) The method of claim 25, wherein the muhimedia types 
include at least one of image, audio, video, synthetic, and text. 

28. (Previously Presented) The method of claim 25, wherein the extracting of 
multimedia objects further comprises: 

segmenting each muhimedia content into segments including content from at 
least one of the multimedia types for the multimedia content; and 
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generating at least one feature description for at least one of the segments by 
feature extraction and annotation; 

wherein the generated multimedia object descriptions comprise the at least one 
feature description for the at least one segment. 

29. (Previously Presented) The method of claim 28, wherein the segments are 
selected from the group consisting of local segments and global segments. 

30. (Previously Presented) The method of claim 28, further comprising selecting the 
at least one feature description from the group consisting of media, semantic and 
temporal features. 

31. (Previously Presented) The method of claim 30, wherein the media features are 
further defined by at least one feature description selected from the group consisting 
of data location, scalable representation and modality transcoding. 

32. (Previously Presented) The method of claim 30, wherein the semantic features 
are further defined by at least one feature description selected from the group 
consisting of keywords, who, what object, what action, why, when, where and text 
annotation. 

33. (Previously Presented) The method of claim 30, wherein the temporal features 
are further defined by at least one feature description consisting of duration. 

34. (Previously Presented) The method of claim 28, wherein the extracting of 
multimedia objects further comprises: 

generating media object descriptions from the muhimedia segment for one of 
the multimedia types by media object extraction processing; 

generating media object hierarchy descriptions from the generated media 
object descriptions by object hierarchy construction and extraction processing; and 

generating media entity relation graph descriptions from the generated media 
object descriptions by entity relation graph generation processing. 
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35. (Previously Presented) The method of claim 34, wherein generating media object 
descriptions further comprises: 

segmenting the content of each multimedia type in the multimedia object into 
segments within the muhimedia object by media segmentation processing; and 

generating at least one feature description for at least one of the segments by 
feature extraction and annotation; 

wherein the generated media object descriptions comprise the at least one 
feature description for the at least one of the segments. 

36. (Previously Presented) The method of claim 35, further comprising the step of 
selecting the at least one feature description from the group consisting of media, 
semantic and temporal. 

37. (Previously Presented) The method of claim 35, wherein generating media object 
hierarchy descriptions generates media object hierarchy descriptions of the media 
object descriptions based on media feature relationships of media objects represented 
by the media object descriptions. 

38. (Previously Presented) The method of claim 35, wherein generating media object 
hierarchy descriptions generates semantic object hierarchy descriptions of the media 
object descriptions based on semantic feature relationships of media objects 
represented by the media object descriptions. 

39. (Previously Presented) The method of claim 35, wherein generating media object 
hierarchy descriptions generates temporal object hierarchy descriptions of the media 
object descriptions based on temporal features relationships of media objects 
represented by the media object descriptions. 

40. (Previously Presented) The method of claim 35, wherein generating media object 
hierarchy descriptions generates media object hierarchy descriptions of the media 
object descriptions based on relationships of media objects represented by the media 
object descriptions, and wherein the relationships are selected from the group 
consisting of media feature relationships, semantic feature relationships, temporal 
feature relationships, and spatial feature relationships. 
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41. (Previously Presented) The method of claim 35, wherein generating media entity 
relation graph descriptions generates entity relation graph descriptions of the media 
object descriptions based on relationship of the media objects represented by the 
media object descriptions, wherein the relationships are selected from the group 
consisting of media feature relationships, semantic feature relationships, temporal 
feature relationships and spatial feature relationships. 

42. (Previously Presented) The method of claim 26, wherein generating muhimedia 
object hierarchy descriptions generates multimedia object hierarchy descriptions of 
the muhimedia object descriptions based on media feature relationships of muhimedia 
objects represented by the multimedia object descriptions. 

43. (Previously Presented) The method of claim 26, wherein generating muhimedia 
object hierarchy descriptions generates semantic object hierarchy descriptions of the 
muhimedia object descriptions based on semantic feature relationships of muhimedia 
objects represented by the multimedia object descriptions. 

44. (Previously Presented) The method of claim 26, wherein generating multimedia 
object hierarchy descriptions generates temporal object hierarchy descriptions of the 
multimedia object descriptions based on temporal feature relationships of multimedia 
objects represented by the muhimedia object descriptions. 

45. (Previously Presented) The method of claim 26, wherein generating multimedia 
object hierarchy descriptions generates muhimedia object hierarchy descriptions of 
the multimedia object descriptions based on relationships of multimedia objects 
represented by the muhimedia object descriptions, wherein the relationships are 
selected from the group consisting of media feature relationships, semantic feature 
relationships, temporal feature relationships and spatial feature relationships. 

46. (Previously Presented) The method of claim 25, wherein generating entity 
relation graph descriptions generates the entity relation graph descriptions of the 
multimedia object descriptions based on relationships of multimedia objects 
represented by the muhimedia object descriptions, wherein the relationships are 
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selected from the group consisting of media feature relationships, semantic feature 
relationships, temporal feature relationships and spatial feature relationships. 

47. (Previously Presented) The method of claim 25, further comprising receiving and 
encoding the multimedia object descriptions into encoded description information, 
and storing the encoded description information as the at least one description record. 

48. (Previously Presented) The method of claim 26, further comprising combining 
the muhimedia object description, the muhimedia object hierarchy descriptions, and 
the entity relation graph description to form a multimedia description, and receiving 
and encoding the muhimedia description into encoded description information, and 
storing the encoded description information as the at least one description record. 

49. (Previously Presented) The method of claim 47, wherein the encoding comprises 
binary encoding. 

50. (Previously Presented) The method of claim 48, wherein the encoding comprises 
binary encoding. 

51. (Previously Presented) The method of claim 47, wherein the encoding comprises 
the extensible Markup Language (XML) encoding. 

52. (Previously Presented) The method of claim 47, wherein the encoding comprises 
the extensible Markup Language (XML) encoding. 

53. (cancelled) 

54. (New) A computer-readable medium storing instructions for controlling a 
computing device to utilize description records from multimedia content, the 
instructions comprising: 

identifying multimedia types in multimedia content; 

extracting muhimedia objects to generate multimedia object descriptions from 
the multimedia content for at least one of the multimedia types; 
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generating, from the multimedia object descriptions, non-hierarchical entity 
relation graph descriptions for at least one of the muhimedia types, wherein the non- 
hierarchical entity relation graph descriptions are associated with communication 
between muhimedia objects; and 

integrating the multimedia object descriptions and the entity relation graph 
descriptions to generate at least one description record to represent content embedded 
within the muhimedia content. 

55. (New) A computing device that generates description records from multimedia 
content, the computing device comprising: 

a module configured to identify multimedia types in multimedia content; 

a module configured to extract muhimedia objects to generate muhimedia 
object descriptions from the muhimedia content for at least one of the muhimedia 
types; 

a module configured to generate, from the muhimedia object descriptions, 
non-hierarchical entity relation graph descriptions for at least one of the multimedia 
types, wherein the non-hierarchical entity relation graph descriptions are associated 
with communication between multimedia objects; and 

a module configured to integrate the muhimedia object descriptions and the 
entity relation graph descriptions to generate at least one description record to 
represent content embedded within the muhimedia content. 
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